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Summary 

Background 

In  the  second  half  of  the  1990s,  recruiting  became  increasingly 
difficult  for  the  Navy  and,  as  a  result,  the  Service  sharply  increased 
its  use  of  enlistment  bonuses  (EBs)  and  the  Navy  College  Fund 
(NCF).1  Although  the  Service’s  use  of  these  programs  has  moder¬ 
ated  since  the  beginning  of  the  current  decade  (the  weaker  civil¬ 
ian  labor  market  has  improved  recruiting  conditions),  it  seems 
likely  that  the  Service  will  become  increasingly  reliant  on  enlist¬ 
ment  incentives  to  attract  personnel  over  the  long  term.  Many  of 
the  jobs  performed  by  Navy  enlisted  personnel  are  becoming  in¬ 
creasingly  technical,  but  a  growing  proportion  of  the  more  capa¬ 
ble  high  school  students  are  choosing  to  go  on  to  college.  In  addi¬ 
tion,  fewer  young  people  are  being  directed  toward  military  ser¬ 
vice  because  fewer  have  family  members,  or  other  »le  models, 
with  military  experience.  If  the  Service  is  to  attract  a  sufficient 
number  of  technically  capable  recruits  without  increasing  basic 
pay  for  all  enlisted  personnel,  it  will  need  to  make  greater  use  of 
recruiting  incentives  and  other  types  of  flexible  compensation. 

It  is  possible  that  the  Navy  will  also  need  to  make  greater  use  of 
enlistment  incentives  to  guide  recruits  toward  critical  ratings,  off- 
peak  ship  dates,  and  longer  terms  of  service.  The  Navy  has  con¬ 
cluded  that,  in  order  to  compete  for  personnel  with  civilian  em¬ 
ployers,  to  improve  retention,  and  to  enhance  workforce 
efficiency,  it  will  need  to  offer  recruits  greater  discretion  in  select¬ 
ing  both  their  rating  and  the  terms  of  their  enlistment.  The  Ser¬ 
vice’s  Rating  Identification  Engine  (RIDE)  and  Job  and 
Occupational  Interest  in  the  Navy  (JOIN)  initiatives  will  offer 


1  The  budget  for  EBs  increased  from  $17  million  to  just  under  $100  million, 
and  the  number  of  accessions  promised  the  NCF  rose  from  2,000  to  more 
than  10,000. 
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recruits  more  information  about  the  enlistment  options  for  which 
they  are  eligible,  and  will  give  them  greater  choice  during  the  clas¬ 
sification  process.  Under  RIDE/JOIN,  recruits  may  make  different 
choices  about  their  terms  of  enlistment  than  under  the  current  sys¬ 
tem.  If  the  Navy  wishes  to  maintain  the  current  pattern  of  acces¬ 
sions  under  these  new  initiatives,  it  may  have  to  make  greater  use  of 
enlistment  incentives  to  influence  how  recruits  select  their  ratings, 
ship  dates,  and  lengths  of  service. 


Shortcomings  of  existing  research 

As  the  Navy  has  been  expanding  the  ratings  for  which  it  offers 
enlistment  incentives,  and  has  been  using  enlistment  incentives  to 
meet  a  wider  range  of  objectives,  administering  these  incentives 
has  become  increasingly  complex — and  the  need  for  analytical 
work  to  support  enlistment  incentive  policies  has  grown.  Unfortu¬ 
nately,  due  to  limitations  of  the  available  data,  research  has  not 
kept  pace  with  the  Service’s  need  for  policy  analyses,  and  the  Navy 
currently  has  an  incomplete  picture  of  how  these  incentives  influ¬ 
ence  recruit  behavior. 

When  the  Service  began  its  enlistment  incentive  programs  in  the 
early  1980s,  it  offered  incentives  to  only  high-quality  recruits  who 
were  entering  a  handful  of  more  technical  communities.  At  that 
time,  the  Navy  had  good  analytic  support  for  the  administration  of 
these  policies  because  the  Department  of  Defense  (DOD)  had 
commissioned  careful  tests  of  how  enlistment  incentives  affect 
various  aspects  of  recruit  behavior.2  An  important  aspect  of  these 
analyses  was  their  use  of  experiments — recruits  were  assigned  to 
different  types  of  enlistment  incentives  in  what  were  essentially 
random  processes.  Using  experiments  allowed  researchers  to 
separate  the  effects  of  enlistment  incentives  from  all  the  other  fac¬ 
tors  (e.g.,  economic  conditions)  that  can  influence  recruit  behav¬ 
ior.  Moreover,  the  experimental  data  yielded  results  that  were 
both  specific  and  reliable:  the  analyses  provided  the  military  with 
precise  estimates  of  the  amount  of  additional  funding  that  would 
be  necessary  to  attract  a  specified  number  of  high-quality  recruits. 


2  See  Polich,  Dertouzos,  and  Press  (1986)  and  Fernandez  (1982). 
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It  has  been  20  years  since  the  enlistment  incentive  experiments 
were  undertaken,  and  over  this  time  there  have  been  many 
changes  in  the  military  demand  for  Service  personnel,  in  recruit¬ 
ing  processes,  and  in  the  characteristics  of  the  pool  of  potential  re¬ 
cruits.  As  a  result,  it  is  unlikely  that  the  findings  from  the  initial 
enlistment-incentive  experiments  are  valid  in  the  current  recruit¬ 
ing  environment.  More  recendy,  several  researchers  have  used 
nonexperimental  data  to  explore  the  effects  of  enlistment  incen¬ 
tives,  and  their  analyses  have  suggested  that  these  incentives  may 
be  effective  in  routing  recruits  among  ratings,  inducing  personnel 
to  accept  longer  terms  of  service,  persuading  recruits  to  delay 
their  ship  date,  and  reducing  attrition.3 

Several  of  these  researchers,  however,  have  pointed  out  serious 
limitations  in  analyses  of  enlistment  incentives  that  employ  nonex¬ 
perimental  data,  and  I  argue  throughout  this  paper  that  these 
limitations  are  severe.  A  fundamental  issue  is  that,  in  analyses 
based  on  nonexperimental  data,  it  can  be  very  difficult  to  separate 
the  effects  of  enlistment  incentives  from  the  other  important  de¬ 
terminants  of  recruit  behavior.4  One  example  that  I  discuss  in  de¬ 
tail  in  this  report  is  the  difficulty  of  separating  the  effect  of 
classifiers  from  the  influence  of  enlistment  incentives.  There  is 
significant  evidence  that  classifier  effort  has  an  important  impact 
on  recruits’  enlistment  decisions  but,  in  the  great  majority  of  stud¬ 
ies,  this  impact  has  been  erroneously  attributed  to  enlistment  in¬ 
centives.  Other  problems  with  using  nonexperimental  data  can 
arise  from  errors  in  variables,  high  correlation  among  the  values 
of  enlistment  incentives  for  various  ratings,  and/ or  small  variation 
in  enlistment  incentives  over  an  extended  period:  any  of  these 
problems  can  make  it  difficult  to  use  statistics  to  determine  how 
enlistment  incentives  affect  recruit  behavior. 


3  See  Cox  (2003),  Golfin  (2003),  and  Warner,  Simon,  and  Payne  (2001). 

4  Murray  and  McDonald  (1999,  p.  52)  describe  the  shortcomings  of  using 
nonexperimental  data  in  analyses  of  enlistment  supply  and  explicitly  rec¬ 
ommend  that  the  Services  undertake  new  experiments  on  enlistment  bo¬ 
nuses  and  the  Navy  College  Fund  akin  to  those  conducted  in  the  1980s. 
Warner,  Simon,  and  Payne  (2001,  appendix  B)  provide  detailed  descriptions 
of  many  of  the  econometric  problems  associated  with  the  use  of  nonexperi¬ 
mental  data. 
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The  need  for  better  data 

It  is  possible  that  the  recent  analyses  have  forgone  the  use  of  experi¬ 
mental  data  out  of  concern  that  collecting  these  data  might  be  ex¬ 
pensive,  or  could  be  disruptive  of  the  enlistment  process.  However, 
the  Navy  could  substantially  improve  its  understanding  of  enlist¬ 
ment  incentives  by  initiating  a  small  set  of  experiments  that  could 
be  undertaken  without  expanding  the  budget  for  recruiting  re¬ 
sources  and  without  creating  obstacles  to  making  recruiting  goal. 

The  two  types  of  information  that  the  Navy  would  find  most  useful 
in  setting  the  value  of  enlistment  incentives  would  be  how  many 
more  recruits  are  drawn  into  the  Navy  as  a  result  of  increasing  in¬ 
centives  (the  market-expansion  effect)  and  how  many  shift  from 
one  rating  to  another  (cross  rating  effects).  If  the  Service  had  reli¬ 
able  data  on  both  these  effects,  it  would  be  able  to  quickly  and  ef¬ 
fectively  adjust  enlistment  incentives  to  alter  the  number  of  per¬ 
sonnel  entering  the  Service,  and  to  more  precisely  redirect 
recruits  from  less  critical  ratings  to  more  critical  ratings. 

Unfortunately,  it  is  possible  to  obtain  only  one  of  these  types  of  in¬ 
formation  from  experiments  that  the  Navy  could  undertake  by  it¬ 
self.  Gathering  data  on  own-price  supply  responses  would  have  to 
be  undertaken  across  Services  because  such  an  experiment  would 
require  randomly  altering  enlistment  incentives  for  different 
branches  in  different  recruiting  districts  (very  much  as  was  done 
in  the  enlistment  incentive  experiments  conducted  in  the  1980s). 

Experiments  on  cross-rating  effects,  however,  could  be  undertaken 
solely  within  the  Navy,  and  analyses  based  on  these  data  could 
yield  significant  improvements  over  the  current  methods  used  to 
set  enlistment  incentives.  For  example,  if  the  Navy  knew  that 
there  were  strong  cross-rating  effects  between  the  Aviation  Struc¬ 
tural  Mechanic  (AMS)  rating  and  the  Aviation  Hydraulic  Me¬ 
chanic  (AMH)  rating,  such  that  increasing  the  incentives  for  one 
would  draw  recruits  from  the  other,  the  Service  would  know  that  it 
should  coordinate  changes  in  these  incentives  to  minimize  un¬ 
wanted  borrowing  of  recruits  from  one  rating  to  the  other.  On  the 
other  hand,  if  the  Service  knew  that  there  was  no  such  cross-rating 
effect,  it  could  adjust  the  incentives  for  these  two  ratings  inde¬ 
pendent  of  each  other. 
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The  Navy  could  also  greatly  benefit  from  implementing  experi¬ 
ments  to  determine  the  effect  that  classifiers  havfe  on  channeling 
recruits  among  ratings.  Asch  and  Karoly  (1993)  show  that  classifi¬ 
ers  can  exert  more  of  an  impact  on  recruits’  decisions  than  a  siz¬ 
able  enlistment  bonus.  However,  they  also  point  out  that  virtually 
every  study  of  the  enlistment  process  has  failed  to  account  for  the 
effect  of  classifiers,  and  many  have  erroneously  credited  classifiers’ 
influence  to  enlistment  incentives.  An  experiment  could  deter¬ 
mine  the  circumstances  when  classifiers  are  most  effective  in 
channeling  recruits,  and  could  also  predict  the  effects  of  current 
RIDE/JOIN  initiatives  that  might  lessen  classifiers’  control  over 
the  enlistment  process. 

In  this  analysis,  I  offer  detailed  proposals  for  experiments  that 
could  determine  (1)  the  cross-rating  effects  of  enlistment  incen¬ 
tives  and  (2)  the  impact  of  Navy  classifiers  on  directing  recruits’ 
enlistment  decisions.  I  also  demonstrate  that  these  experiments 
could  be  undertaken  at  low  cost  and  with  little  disruption  of  the 
classification  process.  Given  the  sizable  budget  for  enlistment  in¬ 
centives,  the  potential  for  improving  efficiency  in  the  use  of  incen¬ 
tives,  and  the  modest  cost  of  experiments,  it  seems  likely  that 
gathering  better  data  on  enlistment  incentives  could  have  a  very 
high  return  on  investment  for  the  Navy. 


The  time  is  now 

Undertaking  experiments  on  enlistment  incentives  may  now  be 
especially  timely.  The  Deputy  to  the  Chief  of  Naval  Personnel  has 
observed  that,  with  the  recent  improvements  in  the  recruiting  en¬ 
vironment,  the  Service  should  now  undertake  longer  term  studies 
to  disentangle  the  effects  of  various  recruiting  resources.5  More¬ 
over,  Navy  Personnel  Research,  Studies,  and  Technology  (NPRST) 
is  currently  undertaking  broad-based  reforms  of  the  computer  and 
operational  systems  employed  in  the  classification  process,  and 
CNRC  may  be  able  to  “piggyback”  on  these  efforts  to  develop  the 
systems  necessary  to  carry  out  experiments  on  enlistment 


5  Capt.  Scott  Slocum  (USN  Retired),  NIB,  in  an  address  to  CNRC  staff,  Octo¬ 
ber  8,  2002,  Millington,  TN. 
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incentives  and  classification  processes.  Finally,  CNA  believes  that 
the  Navy’s  senior  policy-makers  are  becoming  increasingly  recep¬ 
tive  to  the  idea  of  conducting  experiments:  the  Chief  of  Naval 
Operations,  ADM.  Vem  Clark,  recendy  stated  that  the  Navy  must 
“refine  [its]  requirements;  optimally  allocate  resources;  aggres¬ 
sively  assess,  train,  and  assign  our  people;  and  conduct  focused  ec- 
perimentation  to  rapidly  deliver  new  concepts  and  technologies  to 
the  fleet”  (emphasis  added)  .6 


See  the  January  2003  message  that  accompanied  the  release  of  the  “CNO’s 
Guidance  for  2003.” 
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Background  and  policy  issues 

Changes  in  the  Navy's  use  of  enlistment  incentives 

The  last  10  years 

Over  the  last  decade,  the  Navy  has  greatly  expanded  its  use  of  EBs 
and  the  NCF,  and  has  changed  the  way  it  employs  enlistment  in¬ 
centives.  In  the  early  1990s,  the  Navy  made  modest  use  of  these 
incentives,  principally  for  attracting  and  routing  high-quality  ac¬ 
cessions  into  the  hardest  to  fill  occupations:  most  bonuses  were  of¬ 
fered  to  recruits  entering  high-skill  ratings  who  agreed  to 
extended  terms  of  enlistment  (to  5  or  6  years)  and  who  shipped  in 
peak  season  (i.e.,  June  through  September).  The  Navy  College 
Fund  was  targeted  in  a  similar  fashion. 

In  the  mid-1990s,  the  Service  began  having  more  difficulty  in 
meeting  its  recruiting  goals  and,  as  a  result,  greatly  expanded  its 
enlistment  incentive  programs.  The  Navy  increased  its  budget  for 
enlistment  bonuses  from  less  than  $15  million  in  1994  to  just  un¬ 
der  $100  million  at  the  end  of  the  decade.  This  increase  reflected 
both  larger  EBs  being  offered  to  recruits  and  a  significant  expan¬ 
sion  in  the  number  of  recruits  being  promised  bonuses.  Use  of 
the  NCF  expanded  even  more  rapidly:  between  1993  and  1994, 
the  number  of  recruits  promised  the  NCF  increased  from  2,000  to 
more  than  10,000. 

Longer  term  trends  likely  to  drive  enlistment  incentives 

While  there  has  been  some  recent  moderation  in  the  use  of  incen¬ 
tives,  as  weakening  in  civilian  employment  conditions  has  im¬ 
proved  the  recruiting  environment,  other  factors  are  likely  to  drive 
greater  long-term  use  of  enlistment  incentives.  Among  these  are 
changes  in  the  youth  population  and  the  increasingly  technical 
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nature  of  many  jobs  performed  by  Navy  enlisted  personnel.  Over 
the  last  two  decades,  more  high  school  graduates  have  been  going 
to  college  and  fewer  have  had  role  models  who  have  served  in  the 
military.  At  the  same  time,  the  Navy  has  been  requiring  greater 
technical  capabilities  of  personnel  in  many  enlisted  ratings.  If  the 
Service  is  to  attract  increasingly  scarce  high-quality  recruits  with¬ 
out  raising  basic  pay  for  all  enlisted  personnel,  it  will  have  to  make 
greater  use  of  flexible  compensation  mechanisms,  such  as  recruit¬ 
ing  incentives. 

Changes  that  the  Navy  is  considering  in  its  classification  processes 
could  also  increase  the  Service’s  reliance  on  recruiting  incentives. 
Under  the  RIDE  and  JOIN  initiatives,  the  Navy  will  allow  recruits 
greater  information  about  the  enlistment  incentives  for  which  they 
are  eligible,  and  more  discretion  in  selecting  their  ratings,  their 
ship  dates,  and  the  length  of  their  initial  terms  of  service. 

While  these  reforms  are  expected  to  improve  job  satisfaction  and 
retention,  they  may  have  significant  and  unpredictable  effects  on 
the  classification  process.  At  present,  classifiers  control  the  infor¬ 
mation  that  is  available  to  recruits  and  can  use  this  control  to  steer 
accessions  into  the  most  critical  ratings  and  ship  dates:  a  classifier 
may  reveal  information  on  only  one  or  two  of  the  most  critical  rat¬ 
ings  for  which  a  recruit  is  qualified.  The  only  constraint  on  a  clas¬ 
sifier  doing  this  is  if  the  recruit  refuses  enlistment  unless  he  or  she 
is  offered  additional  choices. 

It  is  possible  that,  if  recruits  were  allowed  greater  discretion  over 
the  classification  process,  they  would  select  a  distribution  of  rat¬ 
ings,  ship  dates,  and  lengths  of  obligation  that  would  be  less  desir¬ 
able  to  the  Navy  than  that  produced  under  the  current  system.  If 
this  were  to  happen,  it  would  be  necessary  to  use  inducements, 
such  as  enlistment  incentives,  to  steer  recruits  into  choices  that  are 
more  appropriate  for  the  Navy’s  needs. 


Greater  complexity  in  administering  enlistment  incentives 

As  the  Navy  has  expanded  its  use  of  enlistment  incentives,  it  has 
offered  EBs  and  the  NCF  to  an  increasingly  diverse  group  of  sail¬ 
ors,  and  has  used  these  incentives  to  meet  an  increasingly  complex 
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set  of  goals.  As  a  result,  it  has  become  a  more  complicated  task  for 
the  Navy  to  set  incentives  at  the  right  level — avoiding  overspend¬ 
ing  (offering  incentives  greater  than  necessary  to  attract  recruits 
and  to  steer  them  into  desired  terms  of  enlistment)  and  under¬ 
spending  (offering  incentives  that  are  insufficient  to  attract  acces¬ 
sions  into  critical  ratings,  smooth  lumps  in  the  training  pipeline, 
or  induce  recruits  to  sign  for  extended  terms  of  service) . 

The  Navy  offices  responsible  for  setting  enlistment  incentives  have 
been  struggling  with  the  increased  complexity  of  their  tasks  and 
have  been  seeking  analytic  support  for  their  efforts.7  One  impor¬ 
tant  issue  with  which  they  have  been  dealing  is  the  changing  char¬ 
acter  of  those  offered  enlistment  incentives.  By  2000,  the  Navy  was 
offering  enlistment  incentives  of  some  type  to  the  majority  of  re¬ 
cruits:  in  a  departure  from  previous  policy,  the  Service  offered  in¬ 
centives  to  many  lower-quality  recruits  and  to  off-peak  accessions.8 
While  the  enlistment  experiments  of  the  early  1980s  yielded  good 
empirical  work  on  how  enlistment  incentives  affect  high-quality 
accessions  who  enter  high-tech  ratings,  these  findings  are  now 
dated  and  there  is  little  recent  information  on  how  other  Navy  re¬ 
cruits  respond  to  EBs  and  the  NCF. 

Navy  policy-makers  are  also  having  to  determine  enlistment  incen¬ 
tives  for  a  larger  number  of  ratings,  and  little  is  known  about  ei¬ 
ther  the  “own-price  response”  or  the  “cross-rating  effects”  for  the 
majority  of  job  classifications.  When  policy-makers  increase  a  set 
of  enlistment  incentives,  they  have  little  idea  about  how  many 
more  recruits  will  enter  a  particular  rating,  and  whether  any 
additional  recruits  who  are  entering  a  rating  are  being  brought 
new  to  the  Service  or  are  being  shifted  from  different  ratings. 


7  The  Commander  Navy  Recruiting  Command  (CNRC)  and  Military  Person- 
nel  and  Plans  Policy  (N1S)  have  commissioned  several  analyses  of  enlistment 
bonuses  over  the  last  few  years.  See  the  sources  listed  in  the  bibliography. 

8  Off-peak  accessions  are  those  who  ship  between  October  and  May.  Typically, 
they  have  been  out  of  school  for  some  time  and  have  worked  in  civilian  em¬ 
ployment.  Studies  have  suggested  that  these  accessions  are  less  likely  than 
peak  season  accessions  to  complete  their  first  term  in  the  Navy.  This  may  be 
because,  for  off-peak  accessions,  the  military  is  probably  not  their  first  choice 
of  career  and  these  personnel  may  be  entering  the  Service  after  having 
problems  holding  civilian  jobs. 
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The  job  of  those  who  set  he  value  of  enlistment  incentives  is  fur¬ 
ther  complicated  by  the  fact  that  they  must  consider  still  other  func¬ 
tions  of  enlistment  bonuses  and  the  NCF.  The  Navy  has  tried  to  use 
EBs  to  encourage  recruits  to  delay  (or  accelerate)  their  accession 
and  to  induce  recruits  to  accept  longer  obligations  for  the  first  term 
of  service.  Moreover,  the  Navy  has  also  been  studying  the  possibility 
that  these  incentives  affect  retention  after  accession.9 

Unfortunately,  as  the  Navy  has  been  expanding  the  uses  to  which  it 
applies  enlistment  incentives,  there  has  not  been  corresponding 
growth  in  analytic  work  to  help  the  Service  administer  these  policies. 
The  most  carefully  constructed  work  on  enlistment  incentives  is  dated: 
this  work  was  based  on  experimental  data  that  aie  now  20  years  old. 
More  recent  studies  of  enlistment  incentives  have  been  based  on  non- 
experimental  data  and,  as  we  will  argue  below,  have  produced  findngs 
that  are  less  reliable  and  that  offer  poor  policy  guidance. 


Better  policy  guidance  will  require  better  data 

When  Congress  first  authorized  the  EB  and  NCF  programs  in  the 
early  1980s,  it  required  that  the  Department  of  Defense  conduct  a 
careful  test  of  how  enlistment  incentives  affect  various  aspects  of  re¬ 
cruit  behavior.  This  requirement  resulted  h  the  “Army  College 
Fund  Experiment”  and  the  “Enlisted  Bonus  Experiment.”10  A  criti¬ 
cal  element  of  these  analyses  was  the  use  of  experiments;  recruits 
were  assigned  different  types  of  enlistment  incentives  in  what  were 
essentially  random  processes.  This  is  important  because  randomly 
generated  enlistment  incentives  are  uncorrelated  with  (unrelated 
to)  any  other  characteristics  of  the  recruit  or  the  recruit’s  circum¬ 
stances.  Experimental  data  allow  researchers  to  examine  the  effects 
of  enlistment  incentives  on  enlistment  decisions  in  isolation  from 
the  many  other  factors  that  can  influence  recruit  behavior. 


9  Many  researchers  have  observed  that  to  get  the  greatest  advantage  from  the 
Navy  College  Fund,  Service  members  must  leave  the  military.  On  the  other 
hand,  Cox  (2002)  reports  findings  that  suggest  that  the  size  of  enlistment 
bonuses  offered  to  recruits  may  be  positively  correlated  with  the  probability 
that  the  recruits  complete  their  first  term  of  service. 

10  See  Polich,  Dertouzos,  and  Press  (1986)  and  Fernandez  (1982) 
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Since  these  initial  studies,  however,  several  researchers  have  at¬ 
tempted  to  analyze  the  effects  of  enlistment  incentives  using  non- 
experimental  data — administrative  data  that  have  been  generated 
from  the  day-to-day  operations  of  the  EB  and  NCF  programs.  The 
use  of  nonexperimental  data  has  produced  several  severe  em¬ 
pirical  problems  for  these  analyses:  endogeneity  bias,  omitted  vari¬ 
ables  bias,  errors  in  variables,  and  other  difficulties.  The  authors 
of  the  recent  research  have  employed  a  number  of  sophisticated 
econometric  methods  to  overcome  the  difficulties  associated  with 
using  nonexperimental  data.  Most  of  these  authors  readily  admit, 
however,  that  the  available  econometric  techniques  have  not  been 
able  to  overcome  the  estimation  problems  associated  with  using 
nonexperimental  data,  and  several  suggest  that  their  empirical 
findings  must  be  viewed  as  inconclusive. 

Among  the  authors  who  have  used  nonexperimental  data,  Murray 
and  McDonald  (1999)  are  particularly  explicit  in  their  discussion  of 
the  limits  of  econometric  tools  in  overcoming  the  difficulties  associ¬ 
ated  with  analyzing  how  enlistment  incentives  affect  enlistment 
supply.  They  write: 

Better  estimates  of  enlisted  supply  will  require  better 
data.  We  recommend  that  the  services  consider  experi¬ 
ments,  akin  to  those  conducted  in  the  1980s,  to  assess  the 
effects  of  enlistment  bonuses  and  educational  benefits. 

Such  experiments  could  insure  independent  variation  in 
enlistment  bonuses  and  education  benefits,  and  hence  al¬ 
low  a  disentangling  of  their  separate  effects. 


Structure  of  the  paper 

In  the  next  sections,  I  discuss  many  of  the  empirical  problems  that 
arise  from  using  nonexperimental  data  to  analyze  the  effects  of 
enlistment  incentives,  describe  some  of  the  econometric  tools  that 
researchers  have  used  in  their  attempts  to  overcome  these  prob¬ 
lems,  and  indicate  the  shortcomings  of  these  tools.  I  also  give  fur¬ 
ther  details  of  the  benefits  and  costs  of  the  two  experiments  that  I 
am  proposing. 
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Problems  with  using  nonexperimental  data 

For  more  than  a  decade,  the  only  policy  guidance  for  those  who 
set  enlistment  incentives  has  come  from  analyses  based  on  nonex¬ 
perimental  data.  These  analyses  have  employed  widely  recognized 
statistical  estimators  and,  in  many  instances,  have  produced  find¬ 
ings  that  seem  both  plausible  and  precise.  Unfortunately,  the  sta¬ 
tistical  estimators  employed  in  these  works  have  significant 
potential  for  bias  when  they  are  applied  to  the  type  of  nonexperi¬ 
mental  data  that  are  available  on  enlistment  incentives.  As  a  re¬ 
sult,  both  the  estimated  effects  of  enlistment  incentives  suggested 
in  these  works,  and  the  statistical  significance  of  these  findings,  are 
likely  to  be  spurious.  Several  authors  of  recent  studies  have  rec¬ 
ognized  the  shortcomings  of  their  analyses  and  have  suggested 
that  conducting  experiments  is  the  only  way  to  ensure  reliable  and 
precise  estimates  of  the  effects  of  enlistment  incentives. 


The  basic  statistical  tool:  regression  analysis 

When  researchers  assess  the  effects  of  enlistment  bonuses,  they  try 
to  answer  a  simple  question:  “how  does  some  aspect  of  recruit  be¬ 
havior  respond  when  the  Service  changes  an  enlistment  incentive, 
holding  all  else  constant. ?”  All  the  analyses  that  have  addressed  this 
question  have  used  a  simple,  but  effective  statistical  technique: 
multivariate  regression  analysis.  This  tool  allows  a  statistician  to 
define  a  dependent  variable  (perhaps  a  zero/one  measure  that 
indicates  if  a  recruit  enters  a  particular  rating)  and  a  set  of  inde¬ 
pendent  variables  (e.g.,  the  value  of  various  enlistment  incentives, 
the  unemployment  rate,  and  demographic  characteristics  of  the 
recruit),  and  to  estimate  how  the  dependent  variable  would  re¬ 
spond  if  there  were  a  change  in  only  one  of  the  independent 
variables. 

This  technique,  however,  can  only  be  applied  in  a  straightforward 
fashion  if  the  data  on  which  it  is  to  be  employed  meet  particular  criteria: 
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•  All  explanatory  variables  are  exogenous — that  is,  the  values  of 
the  explanatory  variables  are  determined  independent  of  the 
values  of  the  dependent  variable.  (If  the  values  of  an  explanatory 
variable  are  determined  jointly  with  the  values  of  the  dependent 
variable,  the  explanatory  variable  is  said  to  be  endogenous.) 

•  A  researcher  has  data  on  all  the  factors  that  exert  substantial 
influence  on  the  behavior  that  is  being  analyzed.  (There  are 
no  omitted  variables  that  exert  an  impact  on  the  dependent 
variable  and  that  are  correlated  with  the  regressors.) 

•  The  explanatory  variables  are  measured  with  an  acceptable 
level  of  precision  (there  is  not  a  serious  potential  problem  of 
measurement  error) . 

If  these  criteria  are  not  met,  there  are  various  “work  arounds”  that 
researchers  can  apply  to  multivariate  regression  to  compensate  for 
inadequacies  in  data.  These  remedies,  however,  have  significant 
limitations  and  researchers  need  to  be  aware  that  there  may  be  no 
appropriate  statistical  “fix”  to  dealing  with  bad  data — short  of 
gathering  better  data.  In  this  section,  I  describe  four  problems 
that  arise  from  using  non  experimental  data  in  analyses  of  enlist¬ 
ment  incentives:  endogeneity  bias,  omitted  variables  bias,  errors  in 
variables,  and  intractable  data  structure.  In  the  following  section, 
I  discuss  various  remedies  that  researchers  have  applied  in  at¬ 
tempts  to  compensate  for  the  shortcomings  of  nonexperimental 
data  on  enlistment  incentives,  and  I  argue  that  these  “fixes”  have 
not  been  adequate  to  the  job.  These  fixes  include  the  use  of  in¬ 
strumental  variables,  fixed  effects  models,  and  “signing  the  bias.” 


Simultaneity  (endogeneity)  bias 

Several  of  the  more  recent  studies  of  enlistment  bonuses  have  used 
nonexperimental  data  to  explore  how  groups  of  potential  recruits 
respond  to  changes  in  enlistment  incentives.  For  example,  re¬ 
searchers  have  examined  how  the  value  of  enlistment  incentives 
awarded  in  various  recruiting  districts  affects  the  number  of  people 
accessing  from  these  recruiting  districts.  Group  studies  that  use 
nonexperimental  data  face  particular  statistical  problems  because 
the  causal  relationship  between  enlistment  incentives  and  enlistment 
behavior  can  run  in  either  direction:  recruits’  enlistment  choices 
can  depend  on  the  value  of  enlistment  incentives,  but  the  value  of 
incentives  can  also  be  driven  by  enlistment  behavior. 
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Figure  1  presents  an  example  of  simultaneity  bias.  The  illustration 
suggests  that  when  enlistment  incentives  are  increased,  more  per¬ 
sons  are  likely  to  enlist.  When  the  general  recruiting  environment 
improves,  however,  policy-makers  are  likely  to  reduce  the  value  of 
enlistment  incentives.  If  researchers  were  to  ignore  the  joint  de¬ 
termination  of  enlistment  decisions  and  enlistment  incentives, 
their  findings  might  suggest  that  smaller  incentives  actually  in- 
prove  recruiting  conditions. 


Figure  1.  Simultaneous  determination  of  recruiting  success  and 
enlistment  incentives 


Recruiting  Success  A -  / (Enlistment  Incentives) 


Suppose  we  are  trying  to  estimate  the  effect  of  enlistment 
bonuses  on  the  ability  to  make  goal  (the  left-facing  arrow) .  We 
would  need  to  recognize  the  effect  of  the  recruiting 
environment  on  enlistment  incentives  (the  right-facing  arrow). 
Failing  to  do  so  could  result  in  incorrectly  assigning  both  effects 
to  the  action  of  enlistment  incentives  on  recruiting  success. 


Omitted  (missing)  variables  bias 

Analyses  of  enlistment  incentives  based  on  either  individual  or 
group  data  are  subject  to  a  second  estimation  problem — omitted 
variables  bias.  If  researchers  are  unable  to  include  factors  among 
their  explanatory  variables  that  have  substantial  influence  on 
enlistment  behavior  and  that  are  correlated  with  enlistment  incen¬ 
tives,  they  may  attribute  to  enlistment  incentives  the  effects  of  these 
omitted  variables.  For  example,  during  periods  of  difficult  recruit¬ 
ing,  the  Service  may  increase  a  wide  range  of  recruiting  resources, 
including  advertising,  the  number  of  recruiters,  and  enlistment  in¬ 
centives.  If  a  researcher  is  able  to  include  only  enlistment  incen¬ 
tives  among  his  or  her  explanatory  variables,  the  effects  of  the 
other,  omitted  recruiting  resources  might  be  attributed  to  enlist¬ 
ment  incentives. 
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Almost  all  of  the  analyses  that  have  been  conducted  on  enlistment 
incentives  are  likely  to  have  suffered  from  serious  omitted  variables 
bias.  The  greatest  potential  for  this  type  of  bias  results  from  re¬ 
searchers  having  insufficient  information  on  the  role  of  military 
classifiers  and  on  the  economic  factors  that  determine  young  peo¬ 
ples’  willingness  to  enlist. 

Omitting  the  classification  process 

Asch  and  Karoly  (1993)  provide  strong  evidence  that  classifiers  ex¬ 
ert  a  powerful  influence  over  recruits’  enlistment  decisions:  they 
show  that  classifiers  can  exert  more  of  an  impact  on  recruits’  deci¬ 
sions  than  a  sizable  enlistment  bonus.  However,  almost  every 
study  of  the  enlistment  process  has  failed  to  account  for  the  effect 
of  classifiers  and  it  is  likely  that  many  studies  have  erroneously 
credited  classifiers’  influence  to  enlistment  incentives. 

Classifiers  play  a  central  role  in  ensuring  that  recruits  are  matched 
with  the  ratings,  ship  dates,  and  lengths  of  service  that  provide  the 
greatest  benefit  to  the  Navy.  In  a  very  short  interview  (often  less 
than  15  minutes),  the  classifier  must  determine  the  ratings  for 
which  a  recruit  is  eligible  (based  on  ASVAB  scores  and  the  results 
of  the  medical  exam),  assess  the  recruit’s  career  interests,  and  es¬ 
tablish  when  the  recruit  will  be  available  to  ship.  The  classifier 
then  discusses  with  the  recruit  a  sequence  of  offers — each  of  which 
includes  a  rating,  a  ship  date,  a  term  of  obligation,  and  enlistment 
incentives. 

This  process  is  often  compared  to  negotiating  with  a  retail  sales¬ 
person,  such  as  a  car  dealer:  like  the  dealer,  the  classifier  uses  his 
or  her  control  over  the  sequence  in  which  offers  are  made  to 
guide  a  customer  (the  recruit)  to  the  choices  that  are  most  benefi¬ 
cial  to  the  seller  (the  Navy).  Just  as  manipulating  negotiations  in 
this  way  can  increase  profits  in  retail  trade,  it  can  significantly 
benefit  the  Navy  by  guiding  the  choices  made  by  recruits. 

Unfortunately,  existing  data  do  not  reveal  much  about  how  classi¬ 
fiers  sell  enlistments  to  recruits:  data  taken  from  the  day-to-day 
operations  of  the  EB  and  NCF  programs  do  not  reveal  the  precise 
ratings,  incentives,  and  terms  of  accession  with  which  a  recruit  is 


presented.  A  recruit  might  qualify  for  a  dozen  ratings,  each  of 
these  ratings  might  have  different  values  of  enlistment  incentives 
associated  with  them,  and  each  of  these  incentives  might  vary  by 
both  the  length  of  obligation  and  ship  date.  Nevertheless,  it  is 
possible  that  a  classifier  might  present  this  recruit  with  only  a  sin¬ 
gle  rating,  a  single  term  of  obligation,  a  single  ship  date,  and  a 
single  enlistment  incentive.  The  recruit  might  sign  a  contract 
without  being  aware  of  any  of  the  other  options  from  which  he  or 
she  might  have  chosen.  This  has  serious  implications  for  analyses 
of  enlistment  incentives,  because  failing  to  control  for  the  classifi¬ 
cation  process  can  distort  empirical  results. 


What  biases  result  from  omitting  the  classification  process? 

Omitting  the  classification  process  from  among  the  variables  that  ex¬ 
plain  recruits’  enlistment  decisions  can  result  in  either  understating 
or  overstating  the  financial  effects  of  enlistment  incentives. 


Understating  the  financial  effects  of  incentives 

Some  studies  of  enlistment  incentives  have  implicidy  assumed  that 
recruits  are  presented  with  all  the  ratings  and  all  the  enlistment  in¬ 
centives  for  which  they  are  eligible.  If,  in  actuality,  recruits  are  pre¬ 
sented  with  only  a  portion  of  the  options  for  which  they  qualify, 
these  studies  may  underestimate  the  impact  of  financial  incentives 
on  the  behavior  of  recruits.  To  see  this,  consider  a  researcher  who 
wishes  to  assess  how  a  large  increase  in  the  enlistment  bonus  for  a 
particular  rating  has  affected  the  number  of  recruits  who  entered 
that  rating.  Suppose  that,  unknown  to  this  researcher,  only  a  small 
portion  of  eligible  recruits  were  ever  presented  with  the  rating  or 
told  of  the  large  enlistment  bonus  but  that,  among  those  who  were 
told  of  the  bonus,  a  large  percentage  chose  to  enter  the  rating.  If 
the  researcher  were  to  incorrecdy  assume  that  all  eligible  recruits 
were  told  of  the  rating  and  the  enlistment  incentive,  he  or  she 
might  erroneously  conclude  that  the  bonus  had  only  a  modest  im¬ 
pact  on  the  choices  made  by  recruits. 
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Overstating  the  financial  effects  of  incentives 

Because  classifiers  have  significant  discretion  over  how  they  pre¬ 
sent  ratings,  terms  of  enlistment,  and  incentives,  their  actions  have 
a  large  impact  on  the  choices  made  by  those  entering  the  military. 
The  decisions  that  classifiers  make  are,  in  turn,  formed  by  both 
the  incentives  that  they  face  and  the  information  that  the  Service 
provides  to  them.  One  way  that  the  Navy  communicates  with  clas¬ 
sifiers  is  through  changes  in  enlistment  incentives:  a  sharp  shift  in 
the  enlistment  incentives  offered  for  a  particular  rating,  ship  date, 
or  term  of  obligation  tells  a  classifier  that  the  Navy  wants  recruits 
to  be  steered  in  a  different  direction. 

Classifiers  can  be  expected  to  change  the  options  that  they  “push” 
in  their  interactions  with  recruits  as  a  result  of  changes  in  enlist¬ 
ment  incentives.  In  analyses  based  on  existing  data,  it  is  not  possi¬ 
ble  to  determine  how  much  of  recruits’  response  to  changes  in 
enlistment  incentives  results  from  a  change  in  classifiers  changing 
the  options  that  they  “push”  and  how  much  results  from  changes 
in  the  financial  choices  with  which  they  are  presented. 


Omitting  important  economic  variables 

Analysts  who  explore  how  enlistment  incentives  affect  recruiting 
decisions  often  include  measures  of  labor  market  conditions 
among  their  explanatory  variables.  These  may  include  the  unem¬ 
ployment  rate  for  teenagers  and  a  measure  of  how  civilian  pay 
compares  with  military  pay.  For  several  reasons,  these  measures 
of  labor  market  conditions  can  do  a  poor  job  of  representing  the 
economic  factors  that  recruits  weigh  when  making  enlistment 
decisions. 


Comparing  military  apples  with  civilian  apples 

One  difficulty  arises  from  the  way  economists  evaluate  military  and 
civilian  pay.  The  usual  method  is  to  compare  the  estimated  life¬ 
time  pay  of  Service  members  in  a  particular  rating  with  lifetime 
earnings  of  similar  workers  in  the  private  sector.  For  example,  one 
would  compare  the  career  earnings  of  an  NF  with  earnings  of  civil¬ 
ians  performing  similar  functions  in  a  civilian  nuclear  facility. 
This  is  a  reasonable  method  for  comparing  military  and  civilian 
pay  for  those  who  are  already  firmly  established  in  a  Navy  rating 


and  who,  if  they  were  to  leave  the  Service,  would  likely  take  a  civil¬ 
ian  job  that  is  similar  to  that  which  they  had  in  the  Navy. 

This  approach,  however,  is  less  useful  for  comparing  civilian  and 
military  alternatives  for  those  who  are  completing  high  school  and 
who  are  considering  entering  the  military.  These  people  will  de¬ 
cide  on  military  Service  by  comparing  the  career  pay  for  particular 
ratings  with  a  wide  range  of  civilian  alternatives,  such  as  attending 
college  or  pursuing  a  private  sector  career  for  which  there  may  be 
no  close  substitute  in  the  military. 

The  economic  conditions  of  the  late  1990s  vividly  displayed  the 
shortcomings  of  the  standard  apples-to-apples  approach  of  com¬ 
paring  military  and  civilian  pay.  There  was  a  significant  rise  in  the 
returns  to  attending  college.  Moreover,  many  technically  minded 
high  school  graduates  were  able  to  find  high  paying  positions  in 
such  fields  as  web-page  design  for  e-commerce  firms.  Because 
they  did  not  reflect  the  full  range  of  options  that  were  open  to  the 
recruit-age  population,  the  standard  measures  of  military  and  civil¬ 
ian  pay  may  have  done  a  poor  job  of  capturing  how  the  labor  mar¬ 
ket  of  the  1990s  affected  enlistment  decisions. 


Recruits  care  more  about  immediate  payoffs 

Another  shortcoming  of  the  standard  measures  of  military  and  ci¬ 
vilian  pay  is  that  they  do  not  take  into  account  that  young  peo¬ 
ple — those  in  the  recruit  age  population — often  place  a  very  high 
value  on  near-term  income."  Policy-makers  might  view  a  heating- 
up  of  wages  in  a  period  of  low  unemployment  as  being  a  short¬ 
term  phenomenon  that  would  produce  only  a  modest  increase  in 
lifetime  earnings  in  civilian  employment.  The  Services  might  re¬ 
spond  to  such  a  change  with  a  modest  rise  in  military  basic  pay. 
The  recruit-age  population,  however,  would  likely  be  more  in- 
pressed  with  the  short-term  change  in  civilian  wages  than  with  the 
modest  change  in  basic  pay,  and  would  place  a  greater  value  on  ci¬ 
vilian  employment. 


"  See  Warner  and  Pleeter  (2000) . 
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What  biases  result  from  omitting  economic  variables? 

Since  it  is  not  possible  to  include  adequate  measures  of  the  eco¬ 
nomic  factors  that  drive  recruits’  enlistment  decisions,  it  is  useful 
to  consider  the  direction  of  bias  that  would  result  from  these  un¬ 
observed  variables.  The  scope  and  sign  of  omitted  variables  bias 
will  vary  by  the  type  of  enlistment  behavior  we  are  considering. 
However,  it  seems  likely  that,  in  an  analysis  of  the  market  expan¬ 
sion  effects  of  enlistment  incentives,  omitting  relevant  economic 
variables  would  result  in  understating  the  effects  of  enlistment  in¬ 
centives:  any  unobserved  economic  factors  that  work  against  ac¬ 
cessions  are  likely  to  be  positively  correlated  with  the  value  of 
enlistment  incentives.  As  a  result,  any  positive  effects  that  these 
incentives  have  on  market  expansion  are  likely  to  be  obscured  by 
unobserved  economic  factors. 


Other  problems  with  using  nonexperimental  data 


Errors  in  variables  and  attenuation 

In  addition  to  the  potential  for  endogeneity  bias  and  omitted  vari¬ 
ables  bias,  researchers  who  use  nonexperimental  data  on  enlistment 
incentives  must  also  be  concerned  about  errors  in  the  measurement 
of  enlistment  incentives.  Murray  and  McDonald  (1999)  indicate  that 
poor  measurement  of  the  value  of  enlistment  incentives  is  likely  to 
have  had  severe  effects  on  the  results  of  their  analysis;  they  suggest 
that  data  derived  from  experiments  would  be  less  prone  to  measure¬ 
ment  error.  Golfin  (2003)  provides  a  useful  and  detailed  description 
of  the  many  problems  involved  in  using  administrative  data  to  deter¬ 
mine  the  type  of  enlistment  incentive  that  a  recruit  is  offered. 

In  practice,  the  problem  of  measurement  error  is  very  similar  to 
the  problem  of  omitted  variables.  Greene  demonstrates  that  when 
one  of  the  explanatory  variables  in  a  multiple  regression  analysis  is 
measured  with  error  and  this  error  is  random,  the  estimated  coef¬ 
ficient  on  the  badly  measured  variable  is  biased  toward  zero.12 
The  estimated  coefficients  for  all  of  the  other  variables  in  the  re¬ 
gression  (variables  that  are  measured  without  error)  are  also  bi¬ 
ased,  but  the  sign  of  this  bias  is  unknown. 


12  This  is  known  as  attenuation:  see  Greene  (2000)  p.  378. 


Problems  with  data  structure 

Another  problem  with  using  data  taken  from  the  day-to-day  opera¬ 
tion  of  the  EB  program  is  that  the  Navy  often  implements  lockstep 
adjustments  of  EBs  across  ratings:  when  it  changes  incentives,  the 
Navy  often  adjusts  the  EBs  for  all  ratings  by  some  fixed  propor¬ 
tion.13  This  is  especially  problematic  when  analyzing  the  “cross¬ 
rating  effects”  of  enlistment  bonuses — the  effects  of  changing  the 
value  of  the  incentives  for  one  rating  on  the  number  of  accessions 
going  into  other  ratings.  An  analyst  may  observe  significant  varia¬ 
tion  in  the  EBs  offered  for  a  particular  rating,  but  virtually  no 
change  in  the  relative  value  of  EBs  across  ratings.  As  a  conse¬ 
quence,  he  or  she  may  be  able  to  say  little  about  how  changes  in 
the  relative  value  of  EBs  affect  accessions  across  ratings. 

Similarly,  variation  in  enlistment  incentives  over  extended  periods 
is  often  very  low.  For  example,  among  those  who  accessed  into  the 
nuclear  field  during  the  peak  season  of  FY97,  the  average  EB  was 
$3,300,  but  the  standard  deviation  was  only  $113.  This  lack  of 
variation  hampers  evaluation  of  both  own-price  supply  effects  and 
cross-rating  supply  effects.  These  problems  would  not  exist  with 
experimental  data,  however,  because  one  can  design  into  the  ex¬ 
periment  the  desired  variation  and  correlation  of  incentives. 


13  Golfin  (2003)  discusses  in  detail  how  the  Navy  often  makes  simultaneous,  pro¬ 
portionate  changes  in  the  enlistment  incentives  for  different  ratings.  In  their 
analysis  of  Army  incentives,  Murray  and  McDonald  (1999)  observe  that  “bo¬ 
nuses  and  the  Army  College  Fund  have  generally  been  changed  in  tandem, 
making  it  difficult  to  estimate  their  separate  effects  on  enlistment  behavior.” 
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Tools  for  dealing  with  estimation  problems 

Instrumental  variables 

Various  statistical  techniques  can  be  used  to  try  to  overcome  prob¬ 
lems  of  endogeneity  bias,  omitted  variables  bias,  and  errors  in  vari¬ 
ables.  In  the  recent  analyses  of  enlistment  incentives  that  have 
employed  nonexperimental  data,  the  instrumental  variables  (IV) 
technique  has  been  commonly  used.14  To  apply  this  technique  in 
an  analysis  of  the  effect  of  enlistment  incentives  on  recruit  behav¬ 
ior,  a  researcher  would  have  to  find  some  variable  (or  group  of 
variables)  that  affects  the  size  of  the  enlistment  incentive  that  is  of¬ 
fered  to  a  recruit,  but  that  otherwise  has  no  impact  on  the  choices 
made  by  the  recruit.  As  discussed  in  Cox  (2003),  it  seems  unlikely 
that  such  “identifying  variables”  exist  in  cross-sectional  data:  be¬ 
cause  the  offer  of  an  enlistment  incentive  is  determined  jointly 
with  a  recruit’s  enlistment  decision,  it  seems  improbable  that  there 
could  be  factors  that  would  affect  the  enlistment  incentive  without 
also  affecting  the  enlistment  decision. 

In  an  analysis  of  various  enlistment  incentives  that  employs  panel 
data,  Warner,  Simon,  and  Payne  (2001)  attempt  to  use  IV  to  over¬ 
come  potential  bias  resulting  from  the  endogeneity  of  enlistment 
bonuses  in  panel  data.  The  instrument  that  they  use  is  lagged  val¬ 
ues  of  recruiting  shortfalls:  the  difference  between  contract  mis¬ 
sion  in  a  month  and  the  number  of  contracts  achieved.  The 
authors  point  out,  however,  that  this  approach  is  problematic.  In 
fact,  they  use  this  instrument  only  in  their  analysis  of  Army  data 
and  reject  its  use  in  their  analysis  of  Navy  enlistment  incentives  (in 


14  Davidson  and  MacKinnon  (1993)  provide  a  rigorous  and  very  complete  dis- 
cussion  of  instrumental  variables.  Other  solutions  that  have  been  attempted 
for  errors  in  variables  include  weighted  regression  and  latent  variables  mod¬ 
els  (linear  structural  relations  models):  see  Greene  (2000).  For  an  intuitive 
discussion  of  the  solutions  that  have  been  attempted  for  endogeneity  bias, 
see  Kennedy  (1998). 
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their  analysis  of  the  Navy  data,  they  are  unable  to  find  any  appro¬ 
priate  technique  to  address  the  problems  of  potential  endogeneity 

bias)  .15 


Fixed  effects  models 

A  technique  that  has  been  used  to  reduce  the  potential  for  missing 
variables  is  using  a  “fixed  effects  model”  (or  two-way  fixed  effects 
model).  In  both  Cox  (2003)  and  Warner,  Simon,  and  Payne 
(2001),  this  approach  entails  including  binary  explanatory  vari¬ 
ables  that  represent  the  year  of  accession  and  the  month  of  acces¬ 
sion  (or  the  season  of  accession).  At  first  glance,  it  would  appear 
that  this  approach  would  go  a  long  way  towards  eliminating  the 
potential  for  estimation  bias.  Warner,  Simon,  and  Payne  (2001) 
write  that  college  benefits  and  enlistment  bonuses  “vary  over  time, 
but  not  [across]  state  [s].  By  definition,  then,  neither  of  these 
variables  can  be  correlated  with...  omitted  state  effects.  The  pos¬ 
sibility  that  they  are  correlated  with  omitted  time  effects  is  reduced 
considerably  by  the  inclusion  of  dummy  variables  for  fiscal  year 
and  for  month.” 

The  problem  with  this  approach  is  that  it  assumes  that  the  de  jure 
values  of  enlistment  incentives — the  current  value  of  enlistment 
incentives  as  reported  in  Navy  messages  sent  to  classifiers  at 
MEPS — are  the  values  of  enlistment  incentives  that  are  actually 
presented  to  recruits.  In  a  previous  section,  I  indicated  that  classi¬ 
fiers  are  likely  to  reveal  information  on  only  some  of  the  ratings 


15  There  may  also  be  problems  with  their  use  of  this  instrument  in  their  analy¬ 
sis  of  the  Army  data.  Several  researchers  have  pointed  out  that  recruiting 
goal  is  endogenous;  this  is  discussed  in  Appendix  B  of  Warner,  Simon,  and 
Payne  (2001)  and  in  Berner  and  Daula  (1993).  Despite  the  fact  that  Warner, 
Simon,  and  Payne  (2001)  use  lagged  values  of  recruiting  goal  in  their  instru¬ 
ment  for  enlistment  bonuses,  their  estimations  may  still  be  subject  to  en¬ 
dogeneity  bias.  Deaton  (1997)  writes  “it  is  important  to  realize  that  ... 
simultaneity  cannot  usually  be  avoided  by  using  lags  to  ensure  that  the  right- 
hand  side  variables  are  prior  in  time  to  the  left-hand  side  variables.  If  x  pre¬ 
cedes  y,  then  it  is  reasonable  to  suppose  that  y  cannot  affect  *  direcdy.  How¬ 
ever,  there  is  often  a  third  variable  that  affects  y  today  as  well  as  x  yesterday, 
and  if  this  variable  is  omitted  from  the  regression,  today’s  y  will  contain  in¬ 
formation  that  is  correlated  with  yesterday’s  x.” 
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for  which  a  recruit  is  qualified.  Many  ratings  that  carry  enlistment 
bonuses  are  likely  to  be  mentioned  to  only  a  fraction  of  eligible 
recruits  (this  is  particularly  true  of  ratings  that  are  not  the  most 
critical  and  ratings  that  have  only  a  small  community  of  Service 
members).  When  a  recruit  is  not  informed  of  a  rating  that  carries 
an  enlistment  incentive,  the  de  jure  incentive  is  meaningless  to  the 
recruit. 

In  some  instances,  the  difference  between  the  de  jure  and  de  facto 
values  of  enlistment  incentives  will  be  random — a  simple  conse¬ 
quence  of  different  classifiers  choosing  to  “push”  different  ratings 
to  different  types  of  recruits.  In  such  cases,  the  “errors  in  vari¬ 
ables”  are  likely  to  be  uncorrelated  with  any  important  omitted 
variables,  and  the  estimated  effects  of  enlistment  incentives  will  be 
biased  towards  zero.16 

In  other  situations,  however,  the  difference  between  the  de  jure 
and  de  facto  values  of  enlistment  incentives  may  be  correlated 
with  potentially  important  omitted  variables,  such  as  classifier  ef¬ 
fort.  To  illustrate  this,  recall  that  Navy  Recruiting  Districts  (NRDs) 
are  issued  with  both  goals  and  ceilings  for  the  number  of  recruits 
that  should  be  classified  into  various  high  priority  ratings  and  pro¬ 
grams.  In  an  NRD  that  has  traditionally  hit  its  ceiling  for  a  par¬ 
ticular  rating,  classifiers  are  less  likely  to  discuss  this  rating  with  a 
recruit  or  to  mention  any  enlistment  incentive  associated  with  this 
job.  However,  in  an  NRD  that  has  usually  had  difficulty  hitting  its 
monthly  goal  for  this  rating,  classifiers  could  be  expected  to 
“push”  eligible  recruits  into  this  job.  This  suggests  that  the  differ¬ 
ence  between  the  de  jure  and  de  facto  enlistment  incentives  may 
be  correlated  with  classifier  effort.17 

The  findings  in  Golfin  (2003)  imply  that  there  may  be  many  in¬ 
stances  in  which  the  de  jure  values  of  enlistment  incentives  differ 
from  the  incentives  that  are  actually  offered  to  recruits.  The 


16  Attenuation  bias  (bias  resulting  from  errors  in  measurement  of  explanatory 
variables)  is  discussed  in  a  previous  section. 

17  As  discussed  earlier,  in  analyses  of  enlistment  incentives,  it  is  not  possible  to 
predict  the  sign  of  the  estimation  bias  that  results  from  ignoring  the  impact 
of  classifiers. 
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analysis  shows  that,  among  NRDs,  there  are  large  differences  be¬ 
tween  (1)  the  predicted  proportion  of  recruits  who  access  into  the 
NF  program  and  the  AECF  rating  and  (2)  the  actual  proportion 
who  access  into  these  communities.18  Perhaps  the  most  plausible 
interpretation  of  these  results  is  that,  across  NRDs,  there  are  large 
differences  in  the  proportion  of  eligible  recruits  who  are  pre¬ 
sented  with  a  particular  rating  (and  its  enlistment  incentive). 


Signing  the  bias 

Another  approach  to  analyzing  the  effects  of  enlistment  incentives 
when  facing  the  possibility  of  endogeneity  bias,  omitted  variables 
bias,  or  attenuation  bias  is  to  undertake  standard  multivariate  re¬ 
gression  analysis,  to  recognize  that  the  results  of  this  analysis  may 
be  biased,  and  to  estimate  the  likely  sign  of  this  bias.  When  analyz¬ 
ing  a  continuous  dependent  variable,  one  can  determine  the  bias 
that  would  result  from  omitting  an  important  variable  if  one  is 
confident  about  how  the  omitted  variable  is  correlated  with  the 
dependent  variable  and  how  the  omitted  variable  is  correlated 
with  the  independent  variables  that  are  included  in  the  regression. 
If  one  can  “sign  the  bias”  in  this  way,  regression  results  can  be  in¬ 
terpreted  as  providing  either  a  floor  or  ceiling  for  the  true  rela¬ 
tionship  that  we  wish  to  estimate.  If  a  researcher  can  apply  this 
approach,  he  or  she  could  assert  that  the  true  relationship  be¬ 
tween  two  variables  is  at  least  that  suggested  by  the  regression  re¬ 
sults,  or  at  most  that  suggested  by  the  regression  results.19 


18  These  predictions  are  derived  from  regressing  the  proportion  of  recruits  ac¬ 
cessing  into  these  communities  against  various  personal  characteristics  of 
the  recruits  and  the  unemployment  rate  in  the  recruits’  home  states. 

19  Yatchew  and  Griliches  (1984)  show  that  it  is  typically  rwt  possible  to  sign 
omitted  variables  bias  in  regressions  with  binary  dependent  variables.  Cox 
(2003)  shows  a  narrow  set  of  circumstances  when  it  is  possible  to  sign  estima¬ 
tion  bias  in  regressions  with  binary  dependent  variables. 
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A  proposed  course  of  work 

In  this  section,  I  propose  two  experiments  that  the  Navy  could  un¬ 
dertake  that  would  reveal  important  information  about  the  enlist¬ 
ment  process,  that  would  be  low  cost,  and  that  would  not  hamper 
the  Service’s  ability  to  meet  its  accession  goals.  The  first  of  these 
experiments  would  allow  the  Navy  to  measure  the  effect  that  classi¬ 
fiers  exert  in  directing  recruits  to  terms  of  enlistment,  and  to  de¬ 
velop  better  policies  for  directing  classifiers’  efforts.  (What  litde 
analysis  currently  exists  on  this  topic  suggests  that  classifiers  play  a 
critical  role  in  the  enlistment  process,  but  that  their  influence  is  of¬ 
ten  spuriously  attributed  to  enlistment  incentives.)  The  second  ex¬ 
periment  that  I  suggest  would  allow  the  Service  to  quantify  the 
cross-rating  effects  of  enlistment  incentives.  Such  experiments 
would  give  the  Navy  greater  insight  on  how  it  should  coordinate 
enlistment  incentives  across  different  ratings  and  could  reduce  un¬ 
wanted  “borrowing”  of  recruits  from  one  rating  to  another. 

It  seems  likely  that  there  would  be  large  returns  on  investment  if 
the  Navy  were  to  undertake  these  experiments:  their  costs  are  so 
low  that  they  could  be  recouped  in  a  short  period  with  even  small 
improvements  in  the  management  of  enlistment  incentives.  I  de¬ 
scribe  some  of  the  costs  of  experiments  in  this  section.  I  have  in¬ 
tentionally  kept  these  observations  rather  general  because  it  is  not 
possible  to  give  a  closely  defined  estimate  of  the  costs  of  an  ex¬ 
periment  unless  one  has  decided  exactly  what  question  one  is  trying 
to  answer,  from  what  population  of  individuals  one  is  one  seeking 
this  answer,  and  at  what  level  of  precision  one  requires  the  answer. 
Following  the  description  of  costs,  I  offer  detailed  discussions  of  the 
experiments,  how  they  would  be  carried  out,  and  their  associated 
benefits. 
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The  most  critical  costs 


Advertising  i 

If  DOD  were  to  use  experiments  to  assess  the  effects  of  enlistment 
incentives  on  attracting  personnel  to  the  Navy  (market  expansion 
effects),  it  would  need  to  ensure  that  the  target  population  from 
which  it  draws  its  recruits  would  be  informed  about  the  incentives 
for  enlisting.  If  the  Service  believes  that  its  current  level  of  adver¬ 
tising  is  inadequate  to  inform  potential  recruits  of  expanded  incen¬ 
tives,  it  would  need  to  increase  its  advertising  budget  in  the  markets 
in  which  it  was  conducting  experiments. 

For  most  types  of  experiments,  however,  it  would  not  be  necessary 
to  expand  advertising.  If  the  Navy  were  to  use  an  experiment  to 
assess  the  impact  of  enlistment  incentives  on  inducing  recruits  to 
select  a  specific  rating,  classifiers  could  simply  inform  recruits  of 
different  levels  of  enlistment  incentives  during  the  classification 
process. 

The  budget  for  incentives  and  disruptions  in  classification 

The  central  idea  of  an  experiment  on  enlistment  incentives  is  that 
the  recruits  who  make  up  the  “control  group”  are  offered  one  level 
of  incentive,  while  those  who  are  in  the  “study  group”  are  presented 
with  a  different  level  of  incentive.  On  first  thought,  one  might 
worry  that,  in  order  to  undertake  an  experiment,  the  Service  might 
have  to  bear  the  cost  of  increasing  incentives — this  would  be  the 
case  if  the  incentives  for  the  “control  groups”  were  to  be  kept  at 
their  current  levels  while  those  for  “study  groups”  were  to  be  h- 
creased.  Conversely,  one  might  be  concerned  that  implementing 
an  experiment  could  significandy  hamper  the  Service’s  ability  to 
make  goal:  if  incentives  were  kept  at  their  present  level  in  the 


20  Because  experiments  on  market  expansion  effects  would  require  the  levels 
of  incentives  to  be  advertised,  these  experiments  would  have  to  vary  enlist- 
ment  incentives  across  geographic  regions  and  keep  incentives  consistent 
within  specific  advertising  areas.  When  advertising  is  not  squired  for  an 
experiment,  different  enlistment  incentives  could  be  offered  to  similar  indi¬ 
viduals  within  a  region. 
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control  group  and  decreased  in  the  study  group,  few  recruits  might 
access  from  the  study  group  (or  recruits  in  the  study  group  might 
be  less  inclined  to  the  terms  of  enlistment  that  are  most  advanla- 
geous  to  the  Navy) . 

Many  experiments  could  be  designed,  however,  so  that  the  Navy 
would  not  have  to  expand  its  budget  for  enlistment  incentives  or 
compromise  its  classification  objectives.  The  Service  could  set  the 
level  of  incentive  for  the  control  group  somewhat  below  its  current 
level  and  the  level  of  incentive  for  the  study  group  somewhat 
above  the  current  level.  This  would  be  budget  neutral. 

Under  such  an  arrangement,  it  is  likely  that  there  would  be  little 
change  in  the  overall  distribution  of  enlistment  decisions  among 
recruits.  The  recruits  in  the  control  group  would  be  somewhat  less 
likely  to  select  the  terms  of  enlistment  that  are  favored  by  the 
Navy,  while  those  in  the  study  group  would  be  more  likely  to  choose 
the  preferred  terms  of  enlistment:  in  total,  the  two  effects  could 
more  or  less  offset  each  other. 21 


Setup  costs 

Before  initiating  any  type  of  experiment,  the  Service  would  have 
to  undertake  three  initiatives: 

•  Creating  computer  programs  to  determine  what  offers  are 
made  to  recruits.  The  Navy  collects  data  on  both  the  offers 
for  which  a  recruit  is  qualified  and  the  offers  that  are  pre¬ 
sented  to  a  recruit.  Researchers  who  have  used  these  data 
have  indicated  that  their  structure  is  unwieldy  and  that  evalu¬ 
ating  this  information  would  require  substantial  pro¬ 
gramming  effort.22  The  Service’s  enlistment  software 


21  The  experiments  would  be  budget  neutral  and  would  not  create  obstacles  to 
making  goal  as  long  as  recruits*  responses  to  changing  enlistment  incentives 
were  linear  around  the  initial  level  of  enlistment  incentives;  that  is,  as  long  as 
an  increase  of  $1  in  an  enlistment  incentive  from  its  initial  level  had  exactly 
the  opposite  effect  on  recruits’  behavior  as  a  decrease  of  $1  in  this  incentive. 
Beyond  this  requirement,  it  would  not  matter  whether  recruits  moved  toward 
or  away  from  ratings  that  experience  an  increase  in  enlistment  incentives. 

22  David  Alderton  of  NPRST  provided  information  on  these  data. 
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(OCEAN/RIDE)  should  be  modified  so  that  researchers  can 
more  easily  determine  the  order  in  which  enlistment  options 
are  shown  on  the  computer  screens  presented  to  classifiers. 

•  Creating  computer  algorithms,  to  be  added  to  the  software 
used  during  the  classification  process,  that  would  tell  classifi¬ 
ers  what  ratings  and  terms  of  enlistment  they  should  offer  to 
recruits.  A  central  function  of  these  algorithms  would  be 
adding  some  degree  of  randomization  to  the  enlistment 
choices  that  are  offered  to  recruits. 

•  Identifying  and  implementing  incentives  that  would  ensure 
classifier  participation  in  the  experiments.  Perhaps  the  single 
most  important  element  for  the  success  of  an  experiment 
would  be  ensuring  that  classifier  interactions  with  recruits 
conform  to  the  objectives  of  the  experiment.  Those  who 
administer  Navy  recruiting  would  have  to  ensure  that  the  in¬ 
centives  of  everyone  involved  in  the  classification  process 
were  consistent  with  carrying  out  the  instructions  of  the  ex¬ 
periment.  For  a  discussion  of  classifier  incentive  plans  used 
by  different  branches,  see  Asch  and  Karoly  (1993) . 

Some  of  this  “setup  work”  is  already  underway.  The  CNA  Corpora¬ 
tion  is  undertaking  an  evaluation  of  the  incentives  facing  Navy 
classifiers  and  is  assessing  better  methods  for  informing  classifiers 
of  the  Navy’s  priorities  in  assigning  recruits  to  ratings  and  ship 
dates.23 


An  experiment  on  the  impact  of  classifiers 

Purpose 

The  purpose  of  this  experiment  is  to  assess  the  effectiveness  of 
methods  that  classifiers  use  in  guiding  the  enlistment  decisions  of 
recruits  and  to  implement  improvements  in  these  methods. 


23  The  Classifier  Study,  a  CNAC  initiative  for  CNRC  and  N-l . 
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Rationale  for  analysis 

One  of  the  few  things  that  is  evident  in  the  classifier  process  is  that 
recruits  are  most  likely  to  choose  enlistment  options  from  among 
the  first  several  options  that  are  presented  to  them.  It  is  unclear, 
however,  how  effectively  classifiers  can  persuade  recruits  to  choose 
specific  ratings,  ship  dates,  and  other  terms  of  enlistment.  It  is  also 
unclear  whether  classifiers  are  exerting  their  influence  in  an  effi¬ 
cient  manner.  Classifiers  are  often  dealing  with  difficult  logistics 
and  poor  information:  typically,  they  can  talk  with  a  recruit  for  less 
than  15  minutes,  they  are  able  to  give  recruits  only  sketchy  infor¬ 
mation  on  the  enlistment  options  for  which  they  qualify,  and  they 
operate  with  only  partial  data  on  the  Navy’s  priorities  for  filling 
various  recruiting  slots. 

The  only  credible  way  to  assess  the  influence  of  classifiers  is  with 
an  experiment  that  randomly  assigns  the  sequence  of  offers  that 
are  presented  to  recruits.  For  example,  if  there  were  two  similar 
recruits  who  qualify  for  both  the  x  rating  and  the  y  rating,  classifi¬ 
ers  might  be  instructed  to  promote  x  to  one  and  to  promote  y  to 
the  other.  Without  randomly  determining  the  sequence  of  offers, 
the  first  options  presented  to  a  recruit  would  likely  be  those  that 
the  classifier  believes  will  be  of  greatest  interest  to  the  recruit.  If 
one  cannot  separate  the  sequence  of  offers  from  the  recruit’s  (un¬ 
observed)  intrinsic  interests,  it  is  not  possible  to  assess  the  influ¬ 
ence  that  classifiers  have  over  the  enlistment  process. 


The  scale  of  the  experiment 

Once  the  Service  has  undertaken  the  initial  setup  costs  discussed 
earlier,  experiments  on  the  role  of  classifiers  would  be  “scalable.” 
The  Service  could  start  with  a  very  modest  experiment — perhaps 
involving  several  hundred  personnel — and,  if  it  wished,  could  ex¬ 
pand  this  over  time.  For  example,  the  Service  might  examine 
whether  the  classifiers  at  a  particular  set  of  MEPS  are  effective  in 
influencing  the  enlistment  decisions  of  high  ASVAB  recruits  who 
will  enter  DEP  in  their  last  year  of  high  school.  Over  time,  this 
experiment  could  be  expanded  to  include  other  MEPS  and  other 
classifications  of  recruits. 
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Potential  benefits 

If,  as  seems  likely,  there  is  significant  scope  for  improving  the  way 
classifiers  influence  the  recruiting  process,  the  Navy  may  be  able 
to  gain  substantial  benefit  from  identifying  better  methods  and  in¬ 
centives  for  classifiers.  While  it  is  not  possible  to  estimate  a  precise 
dollar  value  of  these  potential  benefits,  a  rough  frame  of  reference 
might  be  taken  from  Asch  and  Karoly  (199S) .  Their  study  suggests 
that  classifiers  exert  significant  influence  over  recruits’  decisions 
(i)  to  enter  a  particular  rating  and  (ii)  to  accept  a  longer  term  of 
obligation:  their  influence  is  just  as  powerful  as  a  substantial  enlist¬ 
ment  incentive. 


The  costs  of  the  analysis 

This  would  be  a  low-cost  analysis  because  it  would  require  no  in¬ 
crease  in  the  budgets  for  either  enlistment  in  centives  or  advertis¬ 
ing.  Other  than  the  setup  efforts  (discussed  previously),  the 
principal  costs  would  be  for  research  analysis  and  related  data 
programming  support.  Costs  would  be  minimized  if  this  experi¬ 
ment  were  undertaken  over  only  a  single  year,  and  involved  a 
sample  of  relatively  homogeneous  recruits  (recruits  who  had  simi¬ 
lar  ASVAB  scores  and  who  qualified  for  similar  ratings) . 

The  experiment  should  entail  no  interruption  in  the  classification 
process.  The  set  of  options  that  classifiers  would  be  instructed  to 
“push”  would  be  aligned  with  both  classifiers’  incentives  and  the 
Navy’s  requirements  for  routing  personnel  into  various  ratings  and 
ship  dates.  Moreover,  the  experiment  should  not  result  in  any  re¬ 
duction  in  the  number  of  contracts  that  are  completed:  personnel 
who  were  not  receptive  to  the  initial  offers  presented  by  their  clas¬ 
sifier  would  be  shown  alternative  enlistment  options. 


What  would  this  experiment  look  like? 

The  implementation  of  this  experiment  would  be  straightforward. 
Once  results  were  available  from  a  recruits’  ASVAB  and  medical 
exams,  a  computer  algorithm  would  search  among  all  of  the  rat¬ 
ings  for  which  a  recruit  is  eligible  and  would  randomly  pick  one  of 
the  ratings  that  is  most  critical,  together  with  an  available  ship  date 
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for  this  rating.  This  rating  and  ship  date  would  appear  on  the 
classifier’s  computer  monitor.  The  classifier  would  know  that  he 
or  she  would  receive  some  benefit  for  assigning  this  recruit  to  this 
rating  and  ship  date,  but  would  also  face  a  penalty  if  the  recruit 
should  walk  away  without  signing  a  contract.  If  the  recruit  is  not 
willing  to  accept  the  specified  rating  and  ship  date,  the  classifier 
would  offer  other  options. 

What  information  would  this  experiment  yield? 

The  data  generated  from  this  experiment  would  enable  a  re¬ 
searcher  to  assess  how  the  number  of  personnel  assessing  into  a 
particular  rating  could  be  influenced  through  the  “moral  suasion” 
of  the  classifier.  The  experiment  might  identify  particular  types  of 
recruits  or  particular  ratings  for  which  this  sort  of  moral  suasion  is 
particularly  effective. 


An  experiment  on  the  cross-rating  effects  of  incentives 

Purpose 

The  purpose  of  this  experiment  is  to  assess  how  the  number  of  re¬ 
cruits  entering  one  rating  is  affected  by  changes  in  the  value  of 
enlistment  incentives  for  other  ratings. 


Rationale  for  analysis 

Until  the  mid-1990s,  the  Navy’s  principal  objective  in  using  enlist¬ 
ment  incentives  was  to  direct  high-quality  recruits  into  hard-to-fill, 
high-tech  occupations  (e.g.,  the  NF  field  or  the  AECF  rating) .  In 
recent  years,  the  Service  has  greatly  expanded  the  way  it  uses 
enlistment  incentives,  but  the  Navy  continues  to  rely  on  EBs  and 
the  NCF  to  route  recruits  into  critical  ratings.  Despite  its  reliance 
on  this  policy  tool,  the  Service  has  very  incomplete  information 
about  the  way  enlistment  incentives  affect  recruits’  selection  of 
rating. 
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One  open  question  is  especially  important  to  the  Service:  when 
the  Navy  uses  enlistment  incentives  to  shift  recruits  into  a  critical 
rating,  from  where  are  these  recruits  coming?  The  Navy  intends 
that  larger  incentives  will  either  bring  additional  personnel  into 
the  Service  or  shift  recruits  away  from  other,  less  critical  ratings.  It 
is  possible,  however,  that  increasing  the  enlistment  incentive  for 
one  critical  rating  will  simply  shift  recruits  away  from  other  critical 
ratings. 

Failing  to  understand  the  cross-rating  effects  among  various  occu¬ 
pations  could  have  significant  financial  consequences.  To  illus¬ 
trate  this,  suppose  that  enlistment  incentives  for  a  particular  set  of 
critical  ratings  had  weak  market  expansion  effects,  but  strong 
cross-rating  effects.  If  the  Navy  were  unaware  of  these  effects,  it 
might  respond  to  a  difficult  recruiting  environment  by  raising 
enlistment  incentives  in  these  ratings.  This  expansion  of  enlist¬ 
ment  incentives  would  have  little  impact  beyond  redistributing 
personnel  within  this  set  of  critical  ratings. 


To  get  a  useful  picture  of  the  cross-rating  effects  of  enlistment  in¬ 
centives,  one  would  need  to  undertake  a  range  of  different  ex¬ 
periments.  One  reason  for  this  is  that  cross-rating  effects  of  enlist¬ 
ment  bonuses  are  likely  to  differ  among  different  sets  of  ratings: 
changing  enlistment  incentives  for  the  CTI  rating  (linguists)  and 
the  MA  rating  (Master-at-Arms)  would  draw  recruits  from  different 
occupational  groups.  Moreover,  cross-price  effects  may  also  differ 
by  the  personal  characteristics  of  recruits.  For  example,  increas¬ 
ing  the  enlistment  bonus  for  a  particular  rating  might  attract  many 
persons  who  are  overqualified  for  this  occupation,  but  attract 
fewer  personnel  who  are  only  marginally  qualified  (or  vice  versa) . 
Finally,  cross-rating  effects  may  differ  by  the  season  in  which  re¬ 
cruits  enter  DEP  or  by  recruiting  district. 


The  scale  of  the  experiment 

The  Navy  could  begin  experiments  on  cross-rating  effects  on  a 
small  scale — initially  focusing  on  a  few  critical  ratings  and  some  se¬ 
lect  groups  of  recruits.  For  example,  the  Service  could  assess 
whether  offering  increased  enlistment  incentives  to  high-quality, 
peak  season  recruits  entering  the  AECF  rating  would  shift  recruits 
away  from  the  NF  field.  The  Navy  could  later  expand  experiments 


on  cross-rating  effects  to  other  critical  ratings  and  other  groups  of 
recruits. 


Potential  benefits 

The  benefits  of  experiments  on  cross-rating  effects  would  derive 
from  making  more  efficient  use  of  enlistment  incentives.  If  the 
Service  had  full  knowledge  of  both  the  market  expansion  and 
cross-rating  effects  of  enlistment  incentives,  it  could  set  these  in¬ 
centives  with  a  joint  optimization  program:  the  level  of  each 
enlistment  incentive  would  be  increased  until  the  benefit  of  mar¬ 
ket  expansion  effects,  minus  the  monetary  cost  of  the  enlistment 
incentive,  equaled  the  net  cost  (or  benefit)  cf  drawing  personnel 
from  other  ratings.24 

Experiments  on  cross-rating  effects  would  provide  the  Navy  with 
only  part  of  the  information  necessary  for  this  type  of  ideal  opti¬ 
mization  scheme — the  Service  would  know  the  monetary  cost  of 
the  enlistment  incentive  and  the  net  cost  (benefit)  of  drawing  re¬ 
cruits  from  other  ratings.  Having  just  this  partial  information, 
however,  would  permit  significant  improvements  over  the  current 
methods  used  to  set  enlistment  incentives.  For  example,  if  the 
Navy  knew  that  there  were  strong  cross-rating  effects  between  the 
Aviation  Structural  Mechanic  (AMS)  rating  and  the  Aviation  Hy¬ 
draulic  Mechanic  (AMH)  rating,  such  that  increasing  the  incen¬ 
tives  for  one  would  draw  recruits  from  the  other,  the  Service  would 
know  that  it  should  coordinate  changes  in  these  incentives  to 
minimize  unwanted  borrowing  of  recruits  from  one  rating  to  the 
other.  On  the  other  hand,  if  the  Service  knew  that  there  was  no 
such  cross-rating  effect,  it  could  adjust  the  incentives  for  these  two 
ratings  independent  of  each  other. 


The  costs  of  analysis 

The  experiments  could  be  implemented  with  budget  neutral 
changes  in  incentives — recruits  in  the  control  group  would  be  of- 


24  A  net  benefit  (cost)  would  be  created  if  the  cross-rating  effects  of  an  enlist¬ 
ment  incentive  resulted  in  personnel  moving  into  a  rating  from  other  less 
(more)  critical  ratings. 
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fered  an  enlistment  incentive  below  the  current  level  and  those  in 
the  study  group  would  be  offered  an  incentive  above  the  current 
level.  Also,  there  would  be  minimal  disruption  of  the  classification 
process:  the  control  group  would  be  less  likely  to  enter  the  rating 
under  examination,  while  those  in  the  study  group  would  be  more 
likely  to  enter  the  rating.  v 

The  largest  potential  expense  would  be  the  “setup  costs”  discussed 
earlier — the  costs  associated  with  programming,  database  work, 
and  the  implementation  of  appropriate  incentives  for  classifiers. 
However,  if  these  costs  have  already  been  expensed  in  a  previous 
project  (e.g.,  the  classifier  analysis),  they  would  not  have  to  be 
borne  in  the  current  analysis.  The  principal  costs  for  the  current 
initiative  would  be  for  research  analysis  and  a  small  amount  of  pro¬ 
ject  specific  programming. 


What  would  the  experiment  look  like? 

Once  results  are  available  from  a  recruit’s  ASVAB  and  medical  ex¬ 
ams,  a  computer  algorithm  would  search  among  all  of  the  ratings 
for  which  a  recruit  is  eligible  and  would  randomly  pick  one  of  the 
ratings  that  is  most  critical,  together  with  an  available  ship  date  for 
this  rating.  The  computer  program  would  randomly  assign  the  re¬ 
cruit  to  either  the  control  group  (those  who  are  offered  an  enlist¬ 
ment  incentive  smaller  than  that  which  is  currently  offered)  or  the 
study  group  (those  who  are  offered  an  incentive  that  is  larger  than 
that  which  is  currently  offered).  This  rating,  ship  date,  and 
enlistment  incentive  would  appear  on  the  classifier’s  computer 
monitor.  The  classifier  would  know  that  he  or  she  would  receive 
some  benefit  for  assigning  this  recruit  to  this  rating  and  ship  date, 
but  would  also  face  a  penalty  if  the  recruit  should  walk  away  with¬ 
out  signing  a  contract. 
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Conclusion 


In  this  report,  I  argue  that  findings  from  the  analyses  of  enlistment 
incentives  conducted  over  the  last  decade  should  be  considered 
flawed  and  that  the  Navy  should  refrain  from  taking  policy  rec¬ 
ommendations  from  these  works.  The  central  element  of  my  ar¬ 
gument  is  that  there  do  not  exist  adequate  statistical  techniques 
for  dealing  with  the  estimation  problems  that  arise  from  analyzing 
nonexperimental  data  on  enlistment  incentives.  This  assertion  is 
echoed  in  other  works  conducted  by  both  the  Center  for  Naval 
Analyses  and  the  Rand  Corporation.25 

I  propose  two  initiatives  that  the  Navy  could  undertake  that  would 
significantly  improve  the  Service’s  understanding  of  enlistment  in¬ 
centives:  (1)  an  experiment  to  determine  the  cross  rating  effects 
of  enlistment  incentives  and  (2)  an  experiment  to  determine  the 
impact  of  Navy  classifiers  on  directing  recruits’  enlistment  deci¬ 
sions.  I  demonstrate  that  these  experiments  could  be  undertaken 
at  low  cost  and  with  little  disruption  of  the  classification  process, 
and  that  gathering  better  data  on  enlistment  incentives  is  likely  to 
have  a  high  return  on  investment  for  the  Navy. 

Finally,  I  suggest  that  the  time  may  now  be  especially  propitious 
for  undertaking  experiments  on  enlistment  incentives.  NPRST  is 
currently  undertaking  broad-based  reforms  of  the  computer  and 
operational  systems  employed  in  the  classification  process,  and 
CNRC  may  be  able  to  combine  efforts  with  NPRST  to  develop  the 
systems  needed  to  carry  out  experiments  on  enlistment  incentives 
and  classification  processes.  Moreover,  senior  policy-makers 
within  the  Navy  have  recently  voiced  their  support  for  conducting 
experiments  and  for  disentangling  the  effects  of  various  recruiting 
resources. 


25  See  Golfin  (2003)  and  Asch  and  Karoly  (1993). 
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Abbreviations  and  acronyms 

AECF 

Advanced  Electronics  &  Computer  Field 

ASVAB 

Armed  Services  Vocational  Aptitude  Battery 

CNRC 

Commander  Navy  Recruiting  Command 

DOD 

Department  of  Defense 

EB 

Enlistment  Bonus 

JOIN 

Job  and  Occupational  Interest  in  the  Navy 

MEPS 

Military  Enlistment  Processing  Station  (s) 

N13 

Military  Personnel  and  Plans  Policy 

NCF 

Navy  College  Fund 

NF 

Nuclear  Field 

NPRST 

Navy  Personnel  Research,  Studies,  and  Technology 

RIDE 

Rating  Identification  Engine 

ROI 

Return  on  Investment 
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